Configurable Consistency for Wide-area Caching

Data caching is a well-understood technique for improving the performance and availability of wide area distributed applications. The complexity of caching algorithms motivates the need for reusable middleware support to manage caching. To support diverse data sharing needs effectively, a caching middleware must provide a flexible consistency solution that (i) allows applications to express a broad variety of consistency needs, (ii) enforces consistency efficiently among WAN replicas satisfying those needs, and (iii) employs application-independent mechanisms that facilitate reuse. Existing replication solutions either target specific sharing needs and lack flexibility, or leave significant consistency management burden on the application programmer. As a result, they cannot offload the complexity of caching effectively from a broad set of applications. In this dissertation, we show that a small set of customizable data coherence mechanisms can support wide-area replication effectively for distributed services with very diverse consistency requirements. Specifically, we present a novel flexible consistency framework called configurable consistency that enables a single middleware to effectively support three important classes of applications, namely, file sharing, shared database and directory services, and real-time collaboration. Instead of providing a few prepackaged consistency policies, our framework splits consistency management into design choices along five orthogonal aspects, namely, concurrency control, replica synchronization, failure handling, update visibility and view isolation. Based on a detailed classification of application needs, the design choices can be combined to simultaneously enforce diverse consistency requirements for data access. We have designed and prototyped a middleware file store called Swarm that provides network-efficient wide area peer caching with configurable consistency. To demonstrate the practical effectiveness of the configurable consistency framework, we built four wide area network services that store data with distinct consistency needs in Swarm. They leverage its caching mechanisms by employing different sets of configurable consistency choices. The services are: (1) a wide area file system, (2) a proxy-caching service for enterprise objects, (3) a database augmented with transparent read-write caching support, and (4) a real-time multicast service. Though existing middleware systems can individually support some of these services, none of them provides a consistency solution flexible enough to support all of the services efficiently. When these services employ caching with Swarm, they deliver more than 60% of the performance of custom-tuned implementations in terms of end-to-end latency, throughput and network economy. Also, Swarm-based wide-area peer caching improves service performance by 300 to 500% relative to client-server caching and RPCs.